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ABSTRACT 

Mid- continent Research for Education and Learning (McREL) , 
as part of its regular operations, monitors client satisfaction, and seeks to 
sustain a process of informed decision making in the development and delivery 
of products and services. This report documents client satisfaction with 
McREL services as determined by participant evaluations. These evaluations 
were collected at the conclusion of workshops, conferences, or 
consultant/technical assistance provided by McREL staff. The report does not 
include data not collected systematically, such as feedback to staff or 
thank-you letters. A total of 60 workshops and presentations were considered 
as data sources, and, of these, 32 events aligned with the client 
satisfaction study aims. The evaluations after these events indicate that 
clients have been very satisfied with the services they received from McREL 
staff. The grand mean of the 6 dimensions of client satisfaction was 3.47 on 
a 4-point scale, about the same as in 1998, indicating that McREL provided 
high quality services, and was consistent in the delivery of those services. 
(SLD) 
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from the original document. 
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INTRODUCTION 



McREL, as part of its regular operations, monitors client satisfaction in a variety of ways and 
seeks to sustain a process of informed decision making in the development and delivery of products 
and services. This is a second report documenting client satisfaction with McREL services as 
determined by participant evaluations. In our 1998 report, we noted that attentiveness to 
documenting client satisfaction in the non-profit sector has been increasing in recent years. It has 
been a regular concern at McREL with individual event reports prepared and circulated to the 
McREL technical assistance staff provider as well as management. The 1998 report stated: 

“Certainly one aspect of being entrusted with public funds includes delivering 
products and services that are of the highest quality; responding to client needs, and 
providing services and products of direct use to clients and their colleagues” 
(McREL 1999). 

The periodic examination of client satisfaction is one way McREL meets this obligation. This 
report compiles the results of one form of systematic data collection for 1999: participant 

evaluations collected at the conclusion of workshops, conferences, or consultation/technical 
assistance provided by McREL staff. This report does not include non-systematically collected data 
such as oral or written feedback to individual staff, unsolicited feedback received through the 
McREL web site, written testimonials or thank you letters, and measures of product sales or requests 
for services. Given the non-representative nature of this latter type of data and the likely bias 
towards stronger favorable or unfavorable responses by those more motivated to contact McREL, 
these data are seen as less reliable and are not included in this report. 



METHODS 

This section addresses three aspects of the study method: definition, unit of analysis, 
sampling and data collection. 

Definition of Client Satisfaction 



The definition of client satisfaction used in this report is the same as that used for the 1998 
report. Love (1991) suggests that client satisfaction can be measured across three dimensions: “(a) 
the extent to which the service or product has met the consumer’s perceived needs, (b) overall 
consumer satisfaction with the service or product, and (c) whether the consumer would use the 
service or product again (p. 77).” 

Working from Love’s three dimensions of client satisfaction, McREL has identified seven 
subdimensions, which are addressed by the items used for evaluation: 
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(a) meeting the consumers needs - the subdimension enhanced understanding is 
defined; 

(b) overall satisfaction - four subdimensions {material quality, presentation quality, 
amount of information, and overall quality) are defined; and 

(c) whether the client would use the service again - two subdimensions {intend to 
use and share with others) are defined. 

Individual items from specific event evaluation instruments can be aligned with each of the seven 
subdimensions, as shown in Table 1. 



Table 1 

Specifications for Client Satisfaction 



Dimension of 
Client Satisfaction 


Subdimensions 


Example of Item from McREL Survey 


Meeting consumer needs 


Enhanced Understanding 


These workshops enhanced my 
understanding about the readiness and 
model selection process. 

(Extremely [1] to Not at All [4]) 


Overall satisfaction 


Material Quality 


The workshop materials were... 
(Excellent [1] to Poor [4]) 


Presentation Quality 


The workshop presentations and activities 
were... 

(Excellent [1 ] to Poor [4]) 


Amount of Information 


Given the duration of this workshop, the 
amount of information was... 

(Too much [1], About Right [2], Too Little 
[3]) 


Overall Quality 


Overall, this workshop was... 
(Excellent [1] to Poor [4]). 


Whether the client would 
use the service again 


Intend to use (the new 
information/skills) 


I expect to use the information and skills 
acquired during these workshops in my 
work. 

(Extensively [1] to Not at All [4]) 




Intend to share (the new 
inform at ion/ski 11s) 


I expect to share this information with my 
colleagues. 

(Extensively [1] to Not at All [4]) 
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Unit of Analysis 



The unweighted mean of means was calculated to describe each subdimension of client 
satisfaction. The unweighted mean was selected since the event is the unit of analysis and 
participants are nested in events. Responses to items are measured on a four-point scale with 4.0 
being most favorable and 1 .0 being most unfavorable. For one event, a five-point scale was initially 
used, but the results were recalculated to reflect a four-point scale for comparative consistency 
across the entire data set. 

Sampling and Data Collection 

A total of 60 workshops and presentations conducted by McREL under the REL contract in 
fiscal year 1999 were considered as data sources for this report. Of these, evaluation data from 32 
events aligned with the client satisfaction definition given above. Table 2 lists the 32 events that 
served as data sources for this study. As shown in the table, the events occurred throughout the 
entire year and ranged from small (6 participants) to large (171 participants). For each event, the 
McREL staff member providing the service, in conjunction with a member of the McREL evaluation 
team, developed and administered an evaluation form based on a consistent format and core set of 
items. 



Table 2 

FY 1999 Events 



Report Title 


Date 


# of 

Participants 


Comprehensive School Reform: Examining the School 
Readiness Process and Model Selection Process, Missouri 
Department of Elementary and Secondary Education 


October 1998 


156 


Reading in the Content Area 


October 2, 1998 


35 


Southwest Comprehensive Technical Assistance Center: Survey 
of Colorado Clients 


November 9, 1998 


137 


Systems Thinking/Sy stems Changing - Upton, WY 


January 15, 1999 


34 


Dimensions of Learning 


February 12, 1999 


80 


McREL’ s Third Annual Dialogue Group Session, 
“The World Caf6”, facilitated by M. Kellnor-Rogers 


February 17-19, 1999 


30 


Transforming Schoolwides: A Step toward Comprehensive 
School Reform 


February 17, 1999 


30 


Planning Process for School Success, Excellence in Education 
Conference 


April 21, 1999 


11 
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Table 2 (cont’d) 
FY 1999 Events 



Report Title 


Date 


#of 

Participants 


Planning Meeting for Diversity Roundtable 


April 28, 1999 


8 


McREL’s Cross-Site Research Meeting with District and 
University Research Partners 


June 3-4, 1999 


14 


LNP Curriculum, Learning and Instruction (CLI) Project: 
Collaborative Process Evaluation for Project Self-Monitoring and 
Improvement 


June 10-11, 1999 


12 


Symposium on Capacity Building 


June 23-25, 1999 


18 


CSRD Network Forum 


July 27, 1999 


29 


Weston County School District #7 


August 23, 1999 


40 


Systems Thinking/Systems Changing - Sundance, WY 


August 24, 1999 


101 


Dimensions of Learning - Flandreau, SD 


August 24-25, 1999 


100 


McREL All-Staff Retreat 


August 26, 1999 


88 


Regional Field Services Meeting 


September 8, 1999 


6 


Partners Meeting 


September 14, 1999 


25 


Curriculum Learning and Instruction (CLI) Project Team 
Meeting - Portland, OR 


September 27, 1999 


9 


CSRD Regional Meeting 


October 4-7, 1999 


171 


6+1 Writing Assessment Workshop 


October 7-8, 1999 


24 


McREL Policy Forum 


October 11, 1999 


28 


McREL Fall Conference, Pre-session 1 


October 13, 1999 


17 


McREL Fall Conference, Pre-session 2 


October 13, 1999 


39 


McREL Fall Conference, Pre-session 3 


October 13, 1999 


21 


McREL Fall Conference, Pre-session 4 


October 13, 1999 


17 


McREL Fall Conference 


October 13, 1999 


170 


South Dakota CSRD Proposal Planning 


October 27, 1999 


15 


McREL Diversity Roundtable on At-Risk Student Populations 


November 11, 1999 


31 


Performance Indicators/Family Literacy 


November 13, 1999 


15 


CSRD LEA Roundtable 


December 2, 1999 


27 
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FINDINGS 



1999 Client Satisfaction Results 



Table 3 provides the descriptive statistics for each measure of client satisfaction taken in 
1999. Event evaluations are developed jointly between the program presenters and the evaluators 
to best serve the data needs of program improvement as well as overall McREL monitoring of 
quality. As a result not all items reflecting client satisfaction appear on all 32 event evaluation 
instruments; Table 3 also shows the number of samples (events) available for each item. 

Table 3 

Descriptive Statistics by Subdimension 





Event Ratings 


Client Satisfaction Subdimension 


n 


Minimum 


Maximum 


Mean 


Standard 

Deviation 


Enhanced Understanding 


16 


2.86 


3.73 


3.39 


0.27 


Materials Quality 


25 


3.19 


3.85 


3.54 


0.20 


Presentation Quality 


22 


2.85 


3.88 


3.42 


0.27 


Overall Quality 


31 


2.94 


4.0 


3.52 


0.24 


Intended Use 


27 


2.68 


3.87 


3.44 


0.25 


Share with Others 


24 


2.92 


3.80 


3.48 


0.23 


Amount of Information* 


19 


1.70 


2.20 


2.03 


0.12 



*This item was based on a three-point scale where I = Too Much Information, 2 = About the Right Amount of 



Information , and 3 = Too Little Information. 



The overall mean client satisfaction for the 32 events conducted by McREL in 1999 was 3.47 
with a standard deviation of 0.24. This figure does not include the ratings for “ amount of 
information ” item since it is based on a three-point scale. 

Figure 1 provides the means of the 1999 client satisfaction items with their associated 95% 
confidence intervals. The overlapping confidence intervals indicate there is no statistically 
significant difference among the means. In general, across all six subdimensions of client 
satisfaction, clients rate McREL events essentially the same. 
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3.5 



3.3 



Enhanced Understandi 



Presentation Quality 



Materials Quality 



Overall Quality 



Share Wth Others 



Figure 1. Means and 95% confidence intervals for client satisfaction items. 

Comparison of 1998 and 1999 Client Satisfaction 

Table 4 compares the descriptive statistics from the 1 998 and 1 999 client satisfaction reports. 



Table 4 

Descriptive Statistics Comparing Six 1 Client Satisfaction Subdimensions 

from 1998 and 1999 Data 





1998 


1999 


Subdimension 


n 


Mean 


95% Cl 


n 


Mean 


95% Cl 


Enhanced Understanding 


17 


3.41 


3.26/3.56 


15 


3.39 


3.24/3.54 


Materials Quality 


16 


3.54 


3.39/3.70 


25 


3.54 


3.46/3.62 


Presentation Quality 


21 


3.43 


3.30/3.57 


21 


3.42 


3.29/3.55 


Overall Quality 


16 


3.51 


3.36/3.66 


31 


3.52 


3.43/3.61 


Intended Use 


16 


3.34 


3.15/3.53 


26 


3.44 


3.34/3.54 


Share with Others 


15 


3.34 


3.19/3.49 


24 


3.48 


2.49/4.47 



1 The “Amount of Information" item is excluded in this comparison. In 1998, “Amount of Information" was recorded as 
a percentage of those responding u about the right amount of information. ” In 1999, the item was measured as an arithmetic mean 
on the same 1-3 point scale described in Table 3. 
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Ratings in 1999 showed improvements in material quality, and whether clients would share 
what they learned with colleagues ( share with others ), while slight drops in enhanced understanding 
and presentation quality were recorded. In 1998 the mean of all six subdimensions was 3.43; in 
1999 that mean rose slightly to 3.47. In all cases, however, these observed differences are not 
statistically significant. 




□ 1998Data 
Q1999 Data 



Figure 2. Comparison of means for six 1998 and 1999 client satisfaction subdimensions. 



CONCLUSIONS 

As one of several sources of client satisfaction utilized by McREL, the evaluations of 
1999 events indicate that clients have been very satisfied with the services they received from 
McREL staff. In 1999 the grand mean of the six subdimensions of client satisfaction was 3.47 
on a four-point scale, which was about the same as in 1998. In addition to providing high quality 
services in 1999, McREL was consistent in the delivery of those services as means across the 
events were not significantly different. 
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